Add `service list` and `service add` #260

masnax · 2024-02-01T23:43:07Z

Depends on #259 and #287

Adds 2 new commands:

microcloud service list will list the cluster members for every installed service, or report if it is not initialized. This will effectively be the same as calling all of the following in succession:

lxc cluster list
microcloud cluster list
microceph cluster list
microovn cluster list

The information shown will be the name, address, dqlite role, and current status of each member.

microcloud service add will try to setup MicroOVN and MicroCeph on all existing MicroCloud cluster members, optionally setting up storage and networks for LXD. This is useful if MicroOVN or MicroCeph was at one point not installed on the systems and skipped during microcloud init. LXD and MicroCloud itself are required to already be set up.
Thanks to Reuse existing MicroCeph and MicroOVN clusters #259 we can also try to re-use a service that partially covers existing MicroCloud cluster members. So if a MicroCloud is set up without MicroCeph, and then the user manually configures MicroCeph to partially cover the cluster, the user can then use microcloud service add to further configure MicroCeph to work with MicroCloud, and set up storage pools for LXD.

microcloud/service/microceph.go

microcloud/service/microcloud.go

microcloud/service/microovn.go

client/proxy.go

masnax · 2024-05-30T17:27:56Z

@roosterfish @gabrielmougard @markylaing

This PR fails the tests because the microcluster dependency is mismatched between MicroCloud and MicroCeph/MicroOVN. This won't get resolved until the schema issues are fixed on the microcluster side, and the dependency is updated in MicroCeph/MicroOVN.

I thought that since it might take a while, and since the implementation here won't change in the mean time, it would be great if this PR could get some reviews while we wait for it to be unblocked, thanks.

markylaing

Looks good overall, I just have a couple of questions :)

markylaing · 2024-05-31T08:53:18Z

cmd/microcloud/main_init.go

+			err = lxdClient.UpdateProfile(profile.Name, profile.ProfilePut, "")
+			if err != nil {
+				return err
+			}


If the cluster was set up initially with only local storage I'm guessing that when adding ceph afterwards this will overwrite the root disk device in the default profile to be Ceph instead?

I think this might be a bit confusing to users so I'm not sure about it.

Good point, I think we should either ask a question here, or only update the default profile if there is not already a corresponding entry.

Yeah - I guess asking a question makes sense since it's highly likely there will already be root disk and nic devices in the default profile.

api/services.go

roosterfish

Looking very good, I have only a few smaller comments.

I saw #287 is also linked as a dependency but these changes already seem to be included in this PR. Am I right?

client/proxy.go

service/microovn.go

masnax · 2024-06-06T14:23:34Z

Looking very good, I have only a few smaller comments.

I saw #287 is also linked as a dependency but these changes already seem to be included in this PR. Am I right?

Yeah I've just rebased this PR off of that one.

roosterfish · 2024-06-06T14:26:47Z

Yeah I've just rebased this PR off of that one.

Ok. Does this PR require a rebase of some sort? The pipeline doesn't look happy rn.

masnax · 2024-06-06T14:33:00Z

Yeah I've just rebased this PR off of that one.

Ok. Does this PR require a rebase of some sort? The pipeline doesn't look happy rn.

#260 (comment)

The tests won't be happy until MicroCeph & MicroOVN move to the same microcluster version as MicroCloud.

roosterfish · 2024-06-06T14:59:00Z

The tests won't be happy until MicroCeph & MicroOVN move to the same microcluster version as MicroCloud.

My mistake, I have missed the comment. canonical/microcluster#132 is ready :)

To add a service, we need to talk to existing cluster members, so we can use TLS authentication instead of the mDNS auth secret. Signed-off-by: Max Asnaashari <[email protected]>

Signed-off-by: Max Asnaashari <[email protected]>

Rather than overwriting the default profile when adding devices, this appends the devices and config to the default profile. When adding a new service to microcloud that was skipped during initial setup, we may have to add a new device to the default profile but we shouldn't remove anything we added during first init. Signed-off-by: Max Asnaashari <[email protected]>

Adds `microcloud service list` and `microcloud service add` commands. `list` lists all cluster members for every installed and initialized service. `add` is used to add services that were skipped during initialization. MicroCloud's direct cluster members will be compared against MicroCeph and MicroOVN if installed, across all other systems in the cluster. If a cluster exists, the user will be prompted to reuse it. By the end, all cluster members in MicroCloud should also be present in MicroOVN and MicroCeph if chosen. Additionally, if LXD storage pools and networks were not set up initially, they can be set up now. Signed-off-by: Max Asnaashari <[email protected]>

Signed-off-by: Max Asnaashari <[email protected]>

masnax · 2024-06-25T21:03:42Z

@ru-fu I've added the associated documentation to the last commit of this PR, let me know if you have any feedback :)

Signed-off-by: Max Asnaashari <[email protected]>

ru-fu

The docs look good as they are, but I think splitting the instructions into two will make them even clearer - what do you think?

ru-fu · 2024-06-26T13:09:00Z

doc/how-to/add_service.rst

@@ -0,0 +1,41 @@
+.. _howto-add-service:
+
+How to add a new service


Suggested change

How to add a new service

How to add a service

Sorry, pet peeve. ;)
How would you add an old service? 😜

ru-fu · 2024-06-26T13:13:33Z

doc/how-to/add_service.rst

+
+If MicroCloud detects a service is installed but not set up, it will ask to configure the service.
+
+#. Select whether you want to set up distributed storage (if adding MicroCeph to the MicroCloud).


This took me a bit to understand. I think it would be clearer to split the instructions into two parts, one for MicroCeph and one for MicroOVN. The drawback is that we don't have one full list of steps for running the tool, but I think it will still be easy to follow. And users who want to add only one service can skip the part for the other one without wondering if they are missing anything.

Suggested change

#. Select whether you want to set up distributed storage (if adding MicroCeph to the MicroCloud).

To add MicroCeph:

1. Select ``yes`` to set up distributed storage.

ru-fu · 2024-06-26T13:14:28Z

doc/how-to/add_service.rst

+   .. note::
+      To set up distributed storage, you need at least three additional disks on at least three different machines.
+      The disks must not contain any partitions.
+
+   If you choose ``yes``, configure the distributed storage:
+
+   1. Select the disks that you want to use for distributed storage.


Suggested change

.. note::

To set up distributed storage, you need at least three additional disks on at least three different machines.

The disks must not contain any partitions.

If you choose ``yes``, configure the distributed storage:

1. Select the disks that you want to use for distributed storage.

.. note::

To set up distributed storage, you need at least three additional disks on at least three different machines.

The disks must not contain any partitions.

#. Select the disks that you want to use for distributed storage.

Need to fix the indentation for the next steps accordingly.

ru-fu · 2024-06-26T13:15:46Z

doc/how-to/add_service.rst

+
+   #. You can choose to optionally set up a CephFS distributed file system.
+
+#. Select either an IPv4 or IPv6 CIDR subnet for the Ceph internal traffic. You can leave it empty to use the default value, which is the MicroCloud internal network (see :ref:`howto-ceph-networking` for how to configure it).


Is there a reason why this is not a substep to the first one? The question is asked only if you add MicroCeph, correct?

ru-fu · 2024-06-26T13:16:54Z

doc/how-to/add_service.rst

+#. Select whether you want to set up distributed networking (if adding MicroOVN to the MicroCloud).
+
+   If you choose ``yes``, configure the distributed networking:
+
+   1. Select the network interfaces that you want to use (see :ref:`microcloud-networking-uplink`).


Suggested change

#. Select whether you want to set up distributed networking (if adding MicroOVN to the MicroCloud).

If you choose ``yes``, configure the distributed networking:

1. Select the network interfaces that you want to use (see :ref:`microcloud-networking-uplink`).

To add MicroOVN:

1. Select ``yes`` to set up distributed networking.

#. Select the network interfaces that you want to use (see :ref:`microcloud-networking-uplink`).

Plus indentation for the following steps.

ru-fu · 2024-06-26T13:18:55Z

doc/how-to/add_service.rst

+#. MicroCloud now starts to bootstrap the cluster for only the new services.
+   Monitor the output to see whether all steps complete successfully.
+   See :ref:`bootstrapping-process` for more information.


Suggested change

#. MicroCloud now starts to bootstrap the cluster for only the new services.

Monitor the output to see whether all steps complete successfully.

See :ref:`bootstrapping-process` for more information.

MicroCloud now starts to bootstrap the cluster for only the new services.

Monitor the output to see whether all steps complete successfully.

See :ref:`bootstrapping-process` for more information.

masnax requested review from tomponline, roosterfish and markylaing as code owners February 1, 2024 23:43

masnax force-pushed the add-service branch from 23d2c09 to 3ffb8e3 Compare February 1, 2024 23:52

masnax force-pushed the add-service branch from 3ffb8e3 to 1e796d5 Compare April 9, 2024 16:02

github-advanced-security bot found potential problems Apr 9, 2024

View reviewed changes

microcloud/service/microceph.go Fixed Show fixed Hide fixed

microcloud/service/microcloud.go Fixed Show fixed Hide fixed

microcloud/service/microovn.go Fixed Show fixed Hide fixed

masnax force-pushed the add-service branch 3 times, most recently from 9fb1842 to e45632c Compare April 9, 2024 16:37

masnax mentioned this pull request Apr 12, 2024

Reuse existing MicroCeph and MicroOVN clusters #259

Merged

masnax force-pushed the add-service branch 2 times, most recently from 2fa76ca to a4ac008 Compare April 19, 2024 19:38

masnax force-pushed the add-service branch from a4ac008 to a40ca83 Compare May 30, 2024 17:14

github-advanced-security bot found potential problems May 30, 2024

View reviewed changes

client/proxy.go Dismissed Show dismissed Hide dismissed

markylaing reviewed May 31, 2024

View reviewed changes

roosterfish reviewed Jun 6, 2024

View reviewed changes

client/proxy.go Outdated Show resolved Hide resolved

service/microovn.go Outdated Show resolved Hide resolved

masnax force-pushed the add-service branch 3 times, most recently from 606221f to 209ae1c Compare June 12, 2024 00:50

masnax mentioned this pull request Jun 12, 2024

Single node support #319

Merged

4 tasks

roosterfish mentioned this pull request Jun 13, 2024

Add helper for setting up MicroCloud proxy #287

Merged

masnax force-pushed the add-service branch from 209ae1c to bc9f6cb Compare June 15, 2024 00:18

masnax mentioned this pull request Jun 20, 2024

Explicitly set bootstrap flag #327

Closed

masnax added 2 commits June 25, 2024 20:40

microcloud/service: Include TLS cert in LXD remote client

e76be10

To add a service, we need to talk to existing cluster members, so we can use TLS authentication instead of the mDNS auth secret. Signed-off-by: Max Asnaashari <[email protected]>

microcloud/service: Export MicroCloud unix client function

1936502

Signed-off-by: Max Asnaashari <[email protected]>

masnax added 5 commits June 25, 2024 20:40

microcloud/cmd/microcloud: Check existing storage pools before asking

a8b63f5

Signed-off-by: Max Asnaashari <[email protected]>

microcloud/cmd/microcloud: Check existing networks before asking

4baa724

Signed-off-by: Max Asnaashari <[email protected]>

microcloud/test/suites: Add tests for adding services

15825dd

Signed-off-by: Max Asnaashari <[email protected]>

masnax force-pushed the add-service branch 2 times, most recently from cbb67a9 to f17b914 Compare June 25, 2024 21:03

doc/how-to: Add service command docs

f17b914

Signed-off-by: Max Asnaashari <[email protected]>

ru-fu reviewed Jun 26, 2024

View reviewed changes

masnax closed this Jul 23, 2024

roosterfish mentioned this pull request Jul 24, 2024

service add and service list #355

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `service list` and `service add` #260

Add `service list` and `service add` #260

masnax commented Feb 1, 2024 •

edited

Loading

masnax commented May 30, 2024

markylaing left a comment

markylaing May 31, 2024

masnax May 31, 2024

markylaing May 31, 2024

roosterfish left a comment

masnax commented Jun 6, 2024

roosterfish commented Jun 6, 2024

masnax commented Jun 6, 2024

roosterfish commented Jun 6, 2024

masnax commented Jun 25, 2024

ru-fu left a comment

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

ru-fu Jun 26, 2024

		@@ -0,0 +1,41 @@
		.. _howto-add-service:

		How to add a new service


		If MicroCloud detects a service is installed but not set up, it will ask to configure the service.

		#. Select whether you want to set up distributed storage (if adding MicroCeph to the MicroCloud).

-#. Select whether you want to set up distributed storage (if adding MicroCeph to the MicroCloud).
+To add MicroCeph:
+. Select ``yes`` to set up distributed storage.


		#. You can choose to optionally set up a CephFS distributed file system.

		#. Select either an IPv4 or IPv6 CIDR subnet for the Ceph internal traffic. You can leave it empty to use the default value, which is the MicroCloud internal network (see :ref:`howto-ceph-networking` for how to configure it).

Add service list and service add #260

Add service list and service add #260

Conversation

masnax commented Feb 1, 2024 • edited Loading

masnax commented May 30, 2024

markylaing left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

roosterfish left a comment

Choose a reason for hiding this comment

masnax commented Jun 6, 2024

roosterfish commented Jun 6, 2024

masnax commented Jun 6, 2024

roosterfish commented Jun 6, 2024

masnax commented Jun 25, 2024

ru-fu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add `service list` and `service add` #260

Add `service list` and `service add` #260

masnax commented Feb 1, 2024 •

edited

Loading